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Abstract 

This  report  describes  the  equipment  acquired  with  the  support  of  Grant  No.  N00014- 
89-J-1285  to  Brown  University  and  some  of  the  research  projects  carried  out  during 
the  grant  period,  projects  which  have  benefited  firom  the  availability  of  the  scientific 
equipment. 


Accesion  For 

NTIS  CRA&I 
DTIC  TAB 
Unannounced 
Justification 


By . 

Distribution  / 


Availability  Codes 

Oist' 

ft-l 

Avail 

Spe 

and  /  or 
‘Cij| 

□ 

□ 


Statement  A  per  telecon  Julia  Abrahams 

ONR/Code  1111 

Arlington,  VA  22217-5(XX) 

NWW  8/11/92 


1 


I 


Contents 


1  Introduction  1 

2  Laboratory  Configuration  3 

3  Research  Projects  5 

3.1  3D  Object  Recognition .  5 

3.2  Deformable  Templates .  6 

3.3  Computed  Tomography  and  Related  Inverse  Problems .  11 

3.4  Radiation  Therapy  Treatment  Planning .  13 

3.5  Film  Restoration .  17 

3.6  Neural  Networks  .  18 

3.7  Speech  Processing  and  Hidden  Markov  Modeb .  19 


1  Introduction 

The  Grant  titled  Image  Acquisition  and  Processing  Equipment  for  Machine  Vision  (N00014- 
89-J-1285)  funded  under  the  Defense  University  Research  Instrumentation  Program  was 
used  to  obtain  a  sensor  and  state-of-the-art  special-purpose  computer  equipment  for  research 
on  problems  of  image  analysis  and  computer  vision.  The  basic  research  has  been  supported 
by  contracts  with  the  Mathematics  Divisions  of  the  U.S.  Army  Research  Office  and  the  Office 
of  Naval  Research.  Specific  projects  include  ONR  contracts  A  Mathematical  Framework  for 
Image  Analysis  and  Mathematical  Methods  for  Object  Recognition  (just  beginning)  and  ARO- 
supported  A  Renormalization  Group  Approach  to  Image  Processing,  A  New  Computational 
Method  for  3-dimensional  Robot  Vision,  and  The  Complexity  of  the  Cooling  Algorithms  and 
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Center  for  Intelligent  Control  Systems.  Principal  Investigators  of  these  projects  include 
S.  Geman,  B.  Gidas,  U.  Grenander  and  D.E.  McClure  at  Brown  University  and  D.  Geman 
at  the  University  of  Massachusetts  (ONR  subcontract). 

Our  approach  to  problems  in  image  analysis  is  strongly  rooted  in  a  mathematical  sta¬ 
tistical  foundation.  The  foundation  builds  from  Pattern  Theory  and  adopts  a  Bayesian 
formulation  of  image  inference  problems. 

While  a  primary  objective  of  the  research  is  to  develop  a  unified  theoretical  framework  to 
encompass  low-level  image  processing  and  high-level  computer  vision,  the  work  has  contin¬ 
ually  been  driven  by  applications  and  real  data.  Examples  of  motivating  applications  have 
included  enhancement  and  segmentation  of  FLIR  imagery,  texture  segmentations  for  SAR 
imagery,  image  reconstruction  methods  for  single  photon  emission  tomography,  visual  micro¬ 
defect  inspection  of  semiconductor  wafers,  and  identification  of  biological  shapes.  We  always 
bring  the  emerging  theory  back  to  the  motivating  application  and  attempt  to  implement 
algorithms  for  the  driving  problem. 

A  current  thrust  of  our  research  is  on  vision  problems,  broadly  speaking,  the  under¬ 
standing  of  scenes  in  a  three-dimensional  world  based  on  two-dimensional  digital  imagery. 
The  items  funded  by  the  instrumentation  grant  extended  our  abihty  to  study  the  three- 
dimensional  problems.  The  hardware  has  been  installed  in  am  experimental  laboratory  for 
the  acquisition  of  images,  the  computational  analysis  of  image  data  and  of  possible  ap¬ 
proaches  to  vision  problems,  and  the  design  and  implementation  of  parallel  algorithms  for 
computationally  intensive  inference  procedures. 

In  the  next  section,  we  shall  give  a  brief  overview  of  the  equipment  acquired  and  how  the 
laboratory  has  been  configured. 

The  third  section  includes  capsule  summaries  of  completed  and  ongoing  research. 
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2  Laboratory  Configuration 


The  main  equipment  items  proposed  and  acquired  include  (i)  an  imaging  laser  radar  for 
research  experiments  in  3D  object  recognition  from  range  data,  (ii)  a  SUN  host  system  for 
user  control  of  the  laser  radar  and  for  research  in  computer  vision,  (iii)  a  parallel  image  and 
array  processing  board  to  support  research  in  computer  vision,  including  design  of  coarse¬ 
grained  parallel  algorithms,  (iv)  assorted  networking  hardware  and  supplies  for  connecting 
the  laser  radar  system  to  other  research  computing  systems,  and  (v)  a  medium  duty-cycle 
PostScript  printer  for  hard  copy  output  of  image  analysis  experiments.  Each  of  these  items 
is  described  in  detail  below. 

Laser  Radar.  The  laser  radar  is  designed  by  the  Advanced  Intelligent  Machines  Division 
of  Odetics.  It  is  an  imaging  device  that  captures  range  and  amplitude  images  of  a  field  of 
view  of  moderate  scale.  The  amplitude-modulated  imaging  laser  radar  includes  a  scanning 
head  transmitter/receiver,  radar  output  and  digital  storage  device,  and  power  supply.  It  pro¬ 
duces  Digital  Equipment  Corporation  standard  DRVll-W  output  for  128  X  128  reflectance 
(amplitude)  and  range  (phase)  images.  Each  image  subtends  a  60°  x  60°  field  of  view  and 
has  a  30.7-foot  ambiguity  interval,  with  either  1.44-inch  or  0.72-inch  range  resolution.  A 
frame  is  acquired  in  about  O.Ssec. 

SUN  Host  Computer.  The  laiser  radar  (ladau’)  delivers  output  image  data  through  a 
DRVll  interface.  In  order  to  control  the  functions  of  the  laiser  radar  and  to  receive  and  store 
the  acquired  images,  the  ladar  is  connected  via  a  local  area  network  to  a  SUN  host  system. 
The  primary  host  is  a  SUN  4/360  server,  with  VME  bus,  32MB  main  memory,  688MB  disk, 
60MB  cartridge  tape  drive,  19-inch  color  monitor  and  two  client  SPARCstation  I  desktop 
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workstations.  The  SUN  network  provides  immediate  access  for  the  graduate  students,  post¬ 
doctoral  fellows  and  faculty  affiliated  with  our  research  team.  Since  its  initial  acquisition,  the 
SUN  system  has  been  upgraded,  using  funds  from  other  sources,  with  additional  hard  disks, 
a  150MB  tape  drive,  CD  reader,  and  important  software  (Matlab,  APL  II,  compilers,  . . . ). 
The  network  is  interconnected  with  other  research  computer  networks  giving  ready  access  to 
additional  shared  software  and  hardware  resources  (SGI  workstations,  Abekas  Digital  Disk 
Recorder,  VTR  equipment,  Mathematica,  TeX,  . . . ). 

Parallel  Image  and  Array  Processing  Board.  The  SUN  VME-bus  server  also  hosts 
a  vision  performance-enhancement  board.  The  Androx  ICS-400  image  and  array  multi¬ 
processor  board  contains  four  Analog  Devices  ADSP2100  8MHz  digital-signal-processing 
(DSP)  nodes,  each  with  16K  x  24  bits  code  RAM,  16K  x  24  bits  coefficient  RAM,  and  two 
16K-words  data  caches.  The  board  also  has  a  Texas  Instruments  TMS34010  graphics  system 
processor,  which  serves  as  a  data-bus  master  and  gives  the  system  enhanced  graphics  capa¬ 
bilities  in  addition  to  an  ideal  architecture  for  image  analysis.  There  is  SMB  of  high-speed 
video  RAM  (an  increase  from  the  2MB  available  at  the  time  the  proposal  was  prepared), 
which  is  reconfigurable  for  a  variety  of  image  resolutions  and  for  color  images.  Other  fea¬ 
tures  include:  1MB  scratch  memory;  4  input  channels  (RS170,  RS330,  PAL)  with  2K  x  8 
bit  LUTs;  3  output  channels  with  16K  x  8  bit  LUTs.  Software  support  includes  a  C-callable 
image  processing  library  and  DSP  assembler  programming  tools. 

Networking.  Essential  for  the  installation  of  the  ladar  emd  SUN  host  is  networking  hard¬ 
ware  permitting  its  integration  with  other  research  computing  resources.  Network  hardware 
and  supplies  includes  ethernet  tranceivers,  one  multiport  tranceiver,  thinnet  cable,  PC- 
ethernet  cards,  and  SUN  PC-NFS  software. 
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PostScript  Printer.  One  QMS  PS820  TURBO  PostScript  laser  printer  is  available  on  the 
SUN  system.  It’s  main  purpose  is  to  permit  good  quality  half-tone  images  for  work  with  the 
ladar  and  other  image  analysis  work.  Examples  of  the  image  copy  produced  by  the  printer 
are  included  in  §3.1. 

Brown  Uni'^ersity  renovated  and  made  available  two  large  rooms  at  the  main  building  of 
the  Division  of  Applied  Mathematics  for  housing  the  ladar  and  SUN  system.  The  facility  is 
conveniently  located  for  inunediate  access  by  students  and  faculty. 

3  Research  Projects 

In  this  section,  we  describe  some  of  the  completed  and  ongoing  research  projects  which 
have  used  the  equipment  provided  by  this  grant.  The  SUN  host  and  Androx  board  were 
installed  early  in  the  grant  period  (Fall  1989)  and  have  had  a  greater  impact  on  research 
up  to  this  time  than  has  the  laser  radar.  The  laser  radar  was  received  in  late  spring  1990 
and  is  currently  being  configured  for  easy  experimental  use  through  the  implementation  of 
a  convenient,  menu- driven  user  interface. 


3.1  3D  Object  Recognition 

3D  object  recognition  is  a  major  focus  of  a  research  project  commencing  in  October  1990 
and  funded  by  the  Office  of  Naval  Research  [2].  An  abstract  of  the  project  follows: 

Abstract.  Mathematical  Methods  for  Object  Recognition,  Principal  Investigators:  S.  Ge- 
man,  B.  Gidas,  U.  Grenander,  D.E.  McClure,  and  D.  Geman  (subcontract),  ONR  Contract 
N00014-91-J-1021. 
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Probabilistic  models  are  now  widely  used  as  a  basis  for  developing  algorithms  for 
low-level  image  analysis  problems.  This  research  group  has  played  a  leading  role  in 
laying  the  mathematical  foundations  for  the  use  of  Markov  random  held  models  and 
in  demonstrating  the  effectiveness  of  model-based  approaches.  The  proposed  research 
focuses  on  high-level  vision  problems  and  the  development  of  appropriate  mathemat¬ 
ical  frameworks  for  designing  effective  algorithms.  Four  areas  to  be  explored  are: 
rigid  object  recognition,  nonrigid  object  recognition,  two  and  three  dimensional  global 
shape  models,  and  recovery  of  surface  and  shape  information  from  optical  smd  remotely 
sensed  images.  The  mathematical  research  will  be  done  in  combination  with  controlled 
experiments  with  real  data,  including  optical,  laser  radar,  SAR,  and  X-ray  images. 

The  laser  radar  will  be  used  as  one  component  of  the  rigid  body  recognition  work.  We 
shall  study  effective  algorithms  for  the  use  of  range  data  for  object  recognition  and  for  the 
use  of  multi-sensor  data.  Figures  1  and  2  depict  registered  range  and  amplitude  images 
acquired  using  the  ladar.  In  the  range  image,  displayed  intensity  is  proportional  to  distance 
of  a  point  from  the  ladar.  The  amplitude  image  resembles  normal  low-light  video. 

3.2  Deformable  Templates 

Deformable  templates  «u:e  being  developed  by  several  members  of  our  group  as  a  tool  for 
global  analysis  of  images  in  which  objects  with  identifiable  structure,  but  highly  variable 
forms  from  one  instance  to  another,  occur.  Common  examples  are  biological  shapes,  such  as 
a  human  hand;  the  structure  with  a  thumb  and  four  fingers  is  immediately  recognizable,  at 
least  to  a  human  observer,  but  one  individual’s  right  h£uid  can  vary  greatly  from  another’s. 

A  recent  monograph  [8]  lays  the  groundwork  for  probabilistically  deformable  templates. 


Figure  1:  Range  Image  from  Laser  Radar 
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One  recently  completed  project  [1]  and  one  ongoing  project  have  made  extensive  use  of 
the  SUN/ANDROX  system  for  research  on  deformable  templates. 

Abstract.  Structural  Image  Restoration  through  Deformable  Templates,  Y.  Amit,  U,  Gren- 
ander  and  M.  Piccioni,  JASA,  submitted  for  publication. 

Prior  knowledge  on  the  space  of  possible  images  is  given  in  the  form  of  a  function 
or  template  in  some  domain.  The  set  of  all  possible  true  images  is  assumed  to  be 
formed  by  a  composition  of  that  function  with  continuous  mappings  of  the  domain 
into  itself.  A  prior  Gaussian  distribution  is  given  on  the  set  of  continuous  mappings. 

The  observed  image  is  assumed  to  be  a  degradation  of  the  true  image  with  additive 
noise.  Given  the  observed  image,  a  posterior  distribution  is  then  obtained  and  has  the 
form  of  a  nonlinear  perturbation  of  the  Gaussian  measure  on  the  space  of  mappings.  We 
present  simulations  of  the  posterior  distribution  that  lead  to  structural  reconstructions 
of  the  true  image  in  the  sense  that  it  enables  us  to  determine  landmarks  and  other 
characteristic  features  of  the  image,  as  well  as  to  spot  pathologies  in  it.  Moreover, 
we  show  that  the  reconstruction  algorithm  is  relatively  robust  when  the  images  are 
degraded  by  noise  that  is  not  necessarily  additive. 


Abstract.  The  Machine  Recognition  of  Human  Coronary  Arteries,  (Invited  Lectures)  by 
S.  Geman,  J. Elion  and  K.M.  Manbeck,  June  1990. 

Arteriography  is  a  medical  technique  where  an  X-ray  absorbing  dye  is  injected  into 
a  patient’s  arteries.  When  X-rays  are  passed  through  the  patient  and  collected,  a 
medical  specialist  is  able  to  visually  examine  the  data  and  diagnose  abnormalities  such 
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as  blocked  arteries.  Arteriography  is  often  used  by  cardiolo^sts  in  order  to  examine  a 
patient’s  coronary  arteries.  Arteriography  has  proven  to  be  a  valuable  tool,  however, 
one  problem  is  particularly  troublesome.  There  is  a  high  variability  among  cardiologists 
in  the  interpretation  of  arteriograms.  For  this  reason,  it  is  desirable  to  have  a  consistent, 
reliable,  machine  interpretation  of  the  coronary  arteriogram. 

There  are  two  sources  of  variability  present  in  human  coronary  arteries.  The  first 
is  the  natural  variability  one  would  expect  between  different  people,  the  second  comes 
from  the  stretching  and  twisting  that  occurs  as  the  heart  contracts.  Even  though  there 
is  variability,  there  is  also  a  certain  degree  of  invariance.  This  idea  of  variability  super¬ 
imposed  on  an  invariant  structure  brings  to  mind  Grenander’s  notion  of  the  deformable 
template. 

The  notion  of  a  deformable  template  is  exploited  to  design  an  arteriogram  recogni¬ 
tion  package  based  on  a  powerful  mathematical  model.  The  modeling  process  begins 
by  specifying  several  “prototypical”  coronary  artery  structures.  The  prototype  arterial 
structures,  or  templates,  are  studied  in  a  effort  to  quantify  the  variability  between 
patients.  The  prototype  images  are  also  used  to  model  the  effects  of  the  ribs  and  lungs 
which  tend  to  degrade  arteriograms. 

Once  a  workable  mathematical  model  of  an  arteriogram  has  been  developed,  a 
recognition  algorithm  is  applied  to  real  patient  data.  Prototypical  templates  are  de¬ 
formed  into  the  observed  data,  and  the  arteriogram  is  recognized  as  that  template 
which  most  closely  matches  the  data.  The  process  of  determining  the  optimal  match 
is  accomplished  by  a  dynamic  programming  algorithm  similar  to  the  ones  that  have 
been  successfully  implemented  on  the  hidden  Markov  model  approach  to  speech  recog¬ 
nition.  Based  on  the  optimal  template,  the  arteriogram  recognition  package  segments 
and  labels  the  observed  arteriogram  into  its  component  parts. 
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3.3  Computed  Tomography  and  Related  Inverse  Problems 

Previously,  members  of  our  research  group  have  pioneered  the  use  of  Bayesian  statistical 
methods  for  emission  computed  tomography  [5],  [6].  In  the  eaxher  work,  however,  only 
limited  use  was  made  of  data  from  real  clinical  emission-tomography  systems  and  no  ex¬ 
periments  were  carried  out  with  practical  methods  for  accommodating  the  effects  of  photon 
scattering. 

Kevin  Manbeck’s  dissertation  [10]  emphasizes  the  use  of  data  from  real  single-photon- 
emission- tomography  systems  and,  in  his  extensions  of  the  Bayesian  approach,  he  develops 
a  semi-empirical  approach  based  on  well- understood  models  of  the  underlying  physics  for 
compensating  for  effects  of  Compton  scattering.  Computational  experiments,  which  were 
intensive,  made  substantial  use  of  the  SUN/ANDROX  system. 

Abstract.  Bayesian  Statistical  Methods  Applied  to  Emission  Tomography  with  Physical 
Phantom  and  Patient  Data,  by  Kevin  Monroe  Manbeck,  Ph.D.,  Brown  University,  May 
1990. 


In  single  photon  emission  computed  tomography  (SPECT),  projection  data  Y  are 
used  to  reconstruct  an  unknown  isotope  concentration  map  X.  In  this  thesis,  the 
physics  of  SPECT — including  photon  attenuation,  photon  scatter,  and  the  Poisson 
nature  of  radioactive  decay — are  modelled  in  a  probabilistic  framework,  and  a  physical 
phantom  is  used  in  experiments  designed  to  measure  the  magnitude  of  these  effects. 
The  experimental  results  are  incorporated  into  the  entries  of  the  discrete  attenuated 
Radon  transform  A.  The  matrix  A  specifies  the  process  by  which  the  ori^nal  image 
X  is  transformed  into  the  data  Y.  The  model,  studied  from  a  Bayesian  perspective, 
gives  rise  to  a  posterior  probability  distribution  P(X|y).  A  convenient  algorithmic 
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method  of  using  the  data  V  to  estimate  X  is  presented.  The  algorithm,  which  iterates 
conditional  expectations  (ICE)  of  the  posterior  distribution,  is  implemented  in  order 
to  reconstruct  isotope  concentrations.  Isotope  concentrations  axe  estimated  for  both 
patient  data  and  for  physical  phantom  data. 

Building  on  the  work  started  in  this  thesis,  S.  Geman,  K.  Manbeck  and  D.E.  McClure  are 
writing  a  Chapter  for  the  forthcoming  volume  on  theory  and  applications  of  Markov  random 
fields  in  image  analysis  [4].  Computational  experiments  were  done  on  the  SUN/ANDROX 
system. 

Abstract.  A  Comprehensive  Statistical  Model  for  Single  Photon  Emission  Tomography, 
by  S.  Geman,  K.M.  Manbeck  and  D.E.  McClure,  in  Markov  Random  Fields:  Theory  and 
Applications,  R.  Chellappa  and  A.  Jain,  editors,  Academic  Press,  1991(?),  in  preparation. 

A  Bayesian  approach  to  the  reconstruction  of  SPECT  images  is  presented.  The 
approach  requires  two  separate  mathematical  models:  one  model  (called  the  likeli¬ 
hood  or  data  model)  describes  the  process  which  generates  the  data;  the  second  model 
(called  the  prior  model)  captures  certain  prior  knowledge  about  the  image  under  study, 
via  a  probability  distribution  on  the  ensemble  of  possible  reconstructions.  This  paper 
presents  a  general  degradation  model  for  the  SPECT  problem.  In  addition,  experimen¬ 
tal  methods  for  measuring  critical  parameters  in  this  model  are  presented.  Concerning 
the  prior  model,  a  probability  distribution  is  constructed  that  enforces  a  degree  of 
smoothness  while  accommodating  the  discontinuities  associated  with  boundaries.  Ex¬ 
periments  with  physical  phantoms  and  patient  data  demonstrate  the  effectiveness  of  the 
approach.  Finally,  the  isotropic  properties  of  the  prior  model  are  examined  analytically 
in  a  suitable  continuum  limit. 
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Dr.  Majibeck  also  participated  in  research  projects  concerned  with  deblurring  of  Hubble 
telescope  images  [11].  The  work  on  recovering  quantitative  information  about  faint  point 
sources  in  images  affected  by  aberrations  of  the  Hubble  telescope  was  done  in  cooperation 
with  scientists  at  the  University  of  Massachusetts.  The  inverse  problems  encountered  in 
deblurring  the  images  are  formally  similar  to  the  inverse  problems  encountered  in  dealing  with 
Compton  scattering  in  the  context  of  computed  tomography.  Computational  experiments 
were  done  on  the  SUN/ANDROX  system. 

Abstract.  Hubble  Telescope  Image  Restoration  by  Statistical  Methods,  by  K.M.  Manbeck, 
Technical  report,  Division  of  Applied  Mathematics,  Brown  University,  October  1990. 

Two  statistical  methods  are  presented  for  removing  blur  from  Hubble  Space  Tele¬ 
scope  images.  Images  reconstructed  by  maximizing  likelihood  have  some  desirable 
characteristics,  but  suffer  from  a  lack  of  regularization,  as  is  expected.  Images  recon¬ 
structed  in  a  Bayesian  framework  capture  the  necessary  regularization  with  a  prior 
distribution.  The  statistical  methods  are  compared  to  conventional  Fourier  inversion 
techniques  for  deconvolution. 


3.4  Radiation  Therapy  Treatment  Planning 

Only  recently  has  there  been  any  attempt  to  carefully  model  the  physical  processes  in  ra¬ 
diation  therapy  for  the  purpose  of  using  the  models  for  treatment  planning.  The  type  of 
mathematics  that  arises  can  be  interpreted  as  a  dual  version  of  the  tomography  reconstruc¬ 
tion  problem.  In  treatment  planning,  however,  the  system  of  equations  that  describes  a 
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treatment  plan  may  not  have  a  physically  realizable  solution.  Thus  one  is  led  to  the  goal  of 
trying  to  find  a  “best”  approximate  solution  among  those  that  are  physically  realizable. 

Christopher  Raphael’s  thesis  [12]  carefully  formulates  the  treatment  planning  problem  as 
an  optimization  problem.  The  criteria  of  optimality  are  considered  in  terms  of  their  clinical 
interpretations,  as  opposed  to  the  mere  mathematical  convenience.  The  thesis  research  per  se 
is  now  completed.  Extensive  computational  experiments  were  done  on  the  SUN/ANDROX 
system.  Currently,  Raphael  is  writing  and  revising  drafts  of  the  thesis. 

Abstract  (Thesis  Plan).  Mathematics  in  Radiation  Therapy  Treatment  Planning,  by 
Christopher  Raphael,  PhD  Dissertation,  Brown  University,  1991  (expected). 

Ever  since  Marie  Curie’s  discovery  of  the  value  of  radium  in  treating  cancer,  ra¬ 
diation  therapy  has  been  a  field  of  great  importance  in  medicine.  In  its  earlier  days, 
application  techniques  were  crude  and  the  goals  of  treatment  vague.  Since  then,  CAT 
scanning  procedures  have  developed  which  allow  the  precise  differentiation  of  healthy 
tissue  and  tumor.  In  addition  the  modern  teletherapy  machines  enable  the  clinician  to 
deliver  clearly  delineated  and  carefully  contoured  fields  of  radiation.  Now  that  so  much 
more  information  and  control  are  available,  we  feel  acutely  the  need  for  a  mathematical 
context  which  supports  some  notion  of  optimality  in  treatment.  The  main  goal  of  this 
dissertation  is  to  present  this  context  and  examine  a  wide  variety  of  optimization  prob¬ 
lems  ranging  from  the  mathematically  expedient  to  the  most  current  and  biologically 
motivated. 

The  basic  operator  of  this  thesis,  the  dose  operator,  describes  the  relationship 
between  a  treatment  plan  amd  the  dose  distribution  it  produces.  The  treatment  plan 
consists  of  a  collection  of  sources  of  radiation  and  an  associated  collection  of  intensity 
profiles — one  for  each  source.  A  source  can  be  thought  of  as  a  point  lying  somewhere 
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outside  the  patient  while  an  intensity  profile  describes  the  intensities  of  the  various  rays 
which  emanate  from  the  source.  In  practice  this  profile  is  controlled  by  constructing 
a  lead  shield  of  variable  thickness.  Once  the  treatment  plan  has  been  fixed,  various 
physical  phenomena  such  as  attenuation,  divergence,  and  scatter  of  radiation  combine 
to  produce  a  distribution  of  dose  inside  the  patient.  The  dose  operator  which  governs 
this  relationship  is  linear. 

Our  first  chapter  depicts  the  dose  operator  in  the  context  of  jC^.  We  develop  the 
adjoint  operator  which  is  quite  similar  to  the  fundamental  operator  in  tomographic 
medical  imaging.  The  interesting  relationship  between  these  two  operators  leads  to 
some  simple  yet  extremely  useful  identities  which  are  essential  to  the  later  chapters. 
Also  in  this  chapter  we  construct  our  discrete  model  which  is  the  framework  in  which  we 
present  our  computer  experiments.  Due  to  the  generality  of  this  chapter,  the  discrete 
model  is  Just  a  special  case  of  the  theory  already  treated. 

Next  we  treat  the  problem  of  optimization  using  the  ambient  C}  distance  as  a  mea¬ 
sure  of  optimality.  That  is,  we  formulate  our  notion  of  an  “ideal”  dose  distribution 
and  try  to  minimize  the  C?  distance  between  our  potential  treatment  and  this  target. 
The  optimization  is  complicated  by  the  natural  positivity  constraint  on  our  treatments 
since  it  is  not  possible  to  deliver  a  negative  quantity  of  radiation.  We  look  at  this  op¬ 
timization  first  in  the  case  where  the  sources  are  known.  In  this  case  the  dose  operator 
is  linear  and  we  develop  several  interesting  algorithms  which  solve  the  minimization 
and  prove  convergence.  When  the  sources  are  unknown,  the  dose  operator  has  lin¬ 
ear  dependence  on  the  intensity  profiles  but  nonlinear  dependence  on  the  sources.  In 
this  situation  we  have  developed  a  projection-pursuit-like  aJgorithm  for  simultaneously 
choosing  the  sources  and  profiles.  This  algorithm  guarantees  convergence  of  the  asso¬ 
ciated  dose  distributions  to  the  target  when  we  drop  the  positivity  constraint.  In  the 
more  realistic  situation  of  constrained  optimization  with  unknown  sources  we  present 


15 


a  useful  and  reasonable  algorithm  but  provide  no  asymptotic  results. 

Our  next  effort  is  to  consider  optimization  of  more  realistic  measures  of  error  than 
C}  distance.  We  examine  two  of  these.  The  first  formulates  the  problem  as  a  collection 
of  dose  constraints  which  are  not  mutually  satishable.  In  this  setup  the  object  is  to 
satisfy  as  many  constraints  as  possible  (or  more  accurately  a  weighted  sum  over  the 
satisfied  constraints).  The  second  formulation  attempts  to  quantify  the  probability  of 
success  of  a  ^ven  treatment.  This  is  computed  in  terms  of  the  probability  of  tumor 
control  and  the  probability  of  serious  complication,  both  of  which  we  describe  in  terms 
of  simple  and  interesting  biological  models.  For  both  of  these  measures  we  combine 
relaxation  techniques  with  gradient-descent-like  algorithms  to  achieve  our  optimiza¬ 
tion.  These  formulations  provide  encouraging  practical  results  on  the  computer  by 
successfully  choosing  the  “correct”  treatment  philosophy  under  various  assumptions. 

This  thesis  also  examines  some  more  theoretical  issues  pertaining  to  radiation  ther¬ 
apy.  In  one  chapter  we  consider  the  consequences  of  using  the  familiar  Backprojection 
Operator  from  medical  tomography  as  a  model  for  the  dose  operator.  This  assumption 
corresponds  to  disregarding  any  degradation  of  the  beam  as  it  moves  away  from  its 
source  and  assuming  that  the  collection  of  sources  consists  of  the  entire  continuum  of 
possibilities.  We  discuss  the  inversion  theorem  of  the  Radon  Transform  which  gives  our 
ideal  dose  distribution  as  a  backprojection  of  a  certain  treatment.  Unfortunately  this 
treatment  contmns  negative  values  in  general.  We  have  examined  some  possible  com¬ 
promises  although  the  chapter  offers  more  insight  into  the  fundamental  mathematics 
of  radiation  therapy  than  it  offers  practical  solutions. 

A  final  chapter  develops  the  null  spaces  of  several  simple  models  of  the  dose  opera¬ 
tor.  In  particular  we  develop  the  null  spaces  of  the  parallel  beam  model  (no  divergence), 
with  no  attenuation  and  then  with  constant  attenuation.  These  null  sp2u:es  are  useful 
because  they  allow  us  to  modify  a  treatment  without  changing  the  distribution  of  dose 
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it  produces.  This  can  be  used,  for  instance,  to  convert  a  treatment  involving  negative 
values  into  a  more  positive  one,  or  to  distribute  dose  more  evenly  over  the  different 
sources  in  a  treatment. 


3.5  Film  Restoration 

Recently,  work  has  been  done  on  extending  the  Bayesizm  paradigm  for  image  restoration 
to  frame  sequences,  such  as  occur  in  film,  standard  video,  fluoroscopy,  etc.  A  report  is 
currently  being  prepared  [7]  that  shows  what  can  be  achieved  in  using  temporal  as  well 
as  spatial  information  for  removing  various  forms  of  degradation  from  a  frame  sequence. 
Computational  experiments  were  done  on  the  SUN/ANDROX  system. 

Abstract.  A  Nonlinear  Filter  for  Film  Restoration  and  Other  Problems  in  Image  Pro¬ 
cessing,  by  D.  Geman,  S.  Geman  and  D.E.  McClure,  Technical  Report,  Division  of  Applied 
Mathematics,  Brown  University,  1990,  in  preparation. 

A  filter  is  proposed  for  removing  noise  and  other  types  of  degradation.  An  applica¬ 
tion  is  explored  to  enhancement  of  frame  sequences,  motivated  by  the  problem  of  film 
restoration  for  the  movie  industry.  Experiments  are  performed  on  sequences  from  a 
degraded  black-and-white  copy  of  a  recently  released  movie.  Both  temporal  and  spatial 
information  are  used  in  the  restoration.  Temporal  information  is  obtained  from  the 
preceding  unprocessed  frame.  The  smoothing  must  be  nonlinear  in  order  to  preserve 
boundaries. 
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3.6  Neural  Networks 


At  the  Snowbird,  Utah  conference  on  neural  networks  in  April,  S.  Geman  and  E.  Bienenstock 
reported  on  experiments  with  various  learning  algorithms  and  highlighted  the  theoretical 
foundation  that  is  already  in  place  in  the  area  of  nonparametric  statistics.  A  paper  is  in 
preparation  [3],  for  publication  in  Neural  Computation^  reporting  on  further  experiments 
and  on  the  tradeoffs  between  the  number  of  parameters  to  be  learned  and  the  size  of  a 
training  set,  viz.  the  “bias/variance  dilemma”  fanoiliar  in  nonparametric  statistics.  Extensive 
computational  experiments  were  done  on  the  SUN/ANDROX  system. 

Abstract.  Neural  Networks  and  the  Bias/Variance  Dilemma^  by  S.  Geman,  E.  Bienenstock 
and  R.  Doursat,  Technical  Report,  Division  of  Applied  Mathematics,  Brown  University  (to 
be  submitted  to  Neural  Computation)^  1990  (in  preparation). 

Feedforward  neural  networks  trsuned  by  error  backpropagation  axe  examples  of 
nonparametric  regression  estimators.  We  present  a  tutorial  on  nonparametric  inference 
and  its  relation  to  neural  networks,  and  we  use  the  statistical  viewpoint  to  highlight 
strengths  and  weaknesses  of  neural  models.  We  illustrate  the  main  points  with  some 
recognition  experiments  involving  artificiai  data  as  well  as  handwritten  numerals.  In 
way  of  conclusion,  we  suggest  that  current-generation  feedforward  neural  networks  are 
largely  inadequate  for  difficult  problems  in  machine  perception  and  machine  learning, 
regardless  of  parallel-versus-serial  hardware  or  other  implementation  issues.  Further¬ 
more,  we  suggest  that  the  fundamental  challenges  in  neural  modeling  are  about  rep¬ 
resentation  rather  than  learning  per  se.  The  last  point  is  supported  by  additional 
experiments  with  handwritten  numerals. 
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3.7  Speech  Processing  and  Hidden  Markov  Models 

A  question  of  great  interest  and  of  fundamental  importance  for  understanding  how  generic 
is  the  prax;tice  of  using  hidden  Markov  models  (HMMs)  has  recently  been  addressed  by  S. 
Geman,  H.  Kunsch,  and  A.  Kehagias.  They  have  developed  theoretical  results  on  weak 
approximation  of  stationary,  ergodic  stochastic  processes  by  HMMs.  The  results  will  be 
reported  in  the  PhD  dissertation  of  A.  Kehagias  [9]  and  a  subsequent  article.  Kehagias’s 
thesis  plan  describes  the  nature  of  these  results.  Ongoing  computational  experiments  are 
making  extensive  use  of  the  SUN/ANDROX  system. 

Abstract  (Thesis  Plan).  Approximation  of  Stochastic  Processes  by  Hidden  Markov  Mod¬ 
els,  by  Athanasios  Kehagias,  PhD  Dissertation,  Brown  University,  1991  (expected). 

In  this  thesis  we  restrict  ourselves  to  stationary  and  discrete  valued  stochastic 
processes. 

A  pair  of  stochastic  processes  (X,y)  is  a  Hidden  Markov  Model  (HMM)  if  X  (the 
state  process)  is  a  Markov  process  and  Y  (the  observable  process)  is  an  incomplete 
observation  of  X.  The  observation  can  be  deterministic  or  noisy  and  the  observable 
can  be  a  state  or  a  state  transition.  Hence  we  have  four  possible  types  of  HMM’s. 

First  we  establish  that  all  types  of  HMM’s  are  equivalent,  in  the  sense  that,  given 
any  HMM  of  arbitrary  type  we  can  construct  a  HMM  of  any  other  arbitrary  type,  such 
that  the  two  models  have  identical  observable  processes.  Therefore  all  types  of  HMM’s 
have  the  same  modeling  power. 

Second,  we  consider  the  problem  of  Representation:  what  kind  of  stochastic  pro¬ 
cesses  can  we  approximate  with  Hidden  Markov  Models?  To  make  the  question  mean¬ 
ingful  we  define  two  types  of  stochastic  process  approximation:  (a)  weak  approximation, 
based  on  the  weak  convergence  of  probability  measures  and  (b)  cross  entropy  approxi- 
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motion^  based  on  the  Kullback-Leibler  informational  divergence.  Then  we  prove  that 
there  is  a  sequence  of  HMM’s  (of  increasing  size)  that  approximate  any  ergodic  stochas¬ 
tic  process  in  the  weak  and  cross  entropy  sense. 

Third,  we  consider  the  problem  of  Consistent  Estimation.  To  approximate  an  er¬ 
godic  process  we  need  a  sequence  of  HMM’s  of  increasing  size.  For  a  fixed  size  Hidden 
Markov  Model  we  can  use  the  very  efficient  Baum  algorithm  to  find  the  Maximum 
Likelihood  parameters  estimate.  But  will  the  sequence  of  estimates  be  consistent  (i.e. 
will  it  converge  to  the  true  process)?  The  answer  is:  the  sequence  of  Maximum  Likeli¬ 
hood  Estimates  will  be  consistent  if  the  original  process  is  ergodic,  has  strictly  positive 
probability  and  conditional  probability  bounded  away  from  zero. 

Fourth,  we  develop  HMM  models  of  the  raw  speech  signal  and  demonstrate  numer¬ 
ically  consistency  of  Maximum  Likelihood  estimation. 

Finally,  we  develop  Hidden  Gibbs  Models,  an  analogue  of  HMM,  and  use  these  to 
model  one  dimensional  speech  signals  and  two  dimensional  images. 
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